Phoebe: D1 truncation logging + D2 event_count BIGINT (Hugo's decisions)#13
Closed
hhuuggoo wants to merge 2 commits into
Closed
Phoebe: D1 truncation logging + D2 event_count BIGINT (Hugo's decisions)#13hhuuggoo wants to merge 2 commits into
hhuuggoo wants to merge 2 commits into
Conversation
The captured request body is hard-capped before to_tsvector (an uncapped body fails the io_log INSERT past ~1 MiB). That truncation is correct and stays, but it was silent — an operator had no way to see that a stored body was cut. Add a single WARN at the capture call site with the request_id and the original→cap sizes when, and only when, truncation fires. captureRequestBody now also returns the full body length so the caller can log original-vs-cap without re-reading. Behaviour is unchanged otherwise: the forwarded request still carries the full body. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The token-sum and count casts in the rater were already ::bigint, but the rated_usage.event_count COLUMN was still INTEGER and the per-rollup COUNT(*)::int cast still narrowed — an asymmetry that would fail the INSERT on a rollup counting more than 2^31 events. Widen the column (both the raw SQL migration and the Atlas mirror) and the COUNT(*) cast to bigint so the whole path is int64-clean (EventsRated was already int64). No data migration: rated_usage is empty pre-prod, so this is a plain column-type change with no backfill. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
Superseded by the YAML-rater stack (#14→#18). The BIGINT widen (commit a7a74cf) is already in the stack (re-applied in #15+, event_count is BIGINT in the rewritten 0002_rating.sql). The unique survivor — the iolog request-body truncation logging (a8e905f) — has been cherry-picked onto the top of the stack (yaml-rater-fixes-4, now 00833a2). Nothing lost; closing to keep a single clean merge line. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two small, Hugo-approved follow-ups to the just-merged #11/#12. Each is its own commit with a paired test.
D1 — log when the iolog request body is truncated for capture. The captured request body is hard-capped before
to_tsvector(an uncapped body fails theio_logINSERT past ~1 MiB). Hugo's call: keep the truncation, just make it observable. Adds a single WARN at the capture call site withrequest_idand original→cap sizes, firing only when truncation occurs.captureRequestBodynow also returns the full body length so the caller logs without re-reading; forwarded request is unchanged.Test:
TestIOLog_RequestTruncationLogged(over-cap logs the WARN with request_id + sizes; under-cap is silent).D2 — widen
rated_usage.event_countINTEGER → BIGINT. The rater's token-sum and count casts were already::bigint, but theevent_countcolumn was still INTEGER and the per-rollupCOUNT(*)::intcast still narrowed — an asymmetry that would fail the INSERT on a >2^31 rollup. Widens the column (raw0002_rating.sql+ the Atlas mirror), theCOUNT(*)cast, and the integration-test DDL.EventsRatedwas alreadyint64. No data migration —rated_usageis empty pre-prod.Test:
TestIntegration_EventCountIsBigint(asserts the column type is bigint via information_schema AND a >2^31 value round-trips INSERT+SELECT).Full gate green:
go build,go test -race ./...,go vet(+ integration tag), golangci-lint v1.64.8,gofmt -lempty, and the live-Postgres integration suite (-tags=integrationagainst a local pg16).